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Introduction 


The strongest leverage point to improve any system lies with 
the people the system is designed to support (LeFloch, Garcia, 
& Barbour, 2016). Likewise, accountability systems do not move 
schools out of improvement status; educators do. Assessing 
educator performance and using that data to guide their 
professional growth and development holds the potential to 
build and sustain an educator workforce driven by continuous 
improvement so that instructional practice is improved, and 
each student is afforded access to highly effective teachers. 
(LeFloch et al., 2016) Assessing educators’ performance and 
using that information to understand how best to support 


RAND’S RECENT REPORT, A NATIONWIDE LOOK AT TEACHER PERCEPTIONS 
OF FEEDBACK AND EVALUATION SYSTEMS, STATES THAT: 


é G Overall, 76 percent of teachers reported that they 
made improvements to their instructional practices 
as a direct result of their participation in their schools’ 
evaluation systems. 


Teachers that were observed more frequently were more 
likely to report that the evaluation system improved 
their instructional practice. y y 


Tuma, Hamilton, & Tsai, 2018, p. 9. 
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growth and development has the potential to build and 
sustain a workforce that continuously improves practice so 
that all students have equitable opportunity to learn and 
achieve. Schools identified as needing improvement simply 
cannot exit their identified status unless the educators 
within are provided the knowledge, skills, and Supports to 
successfully engage all learners. Educator evaluation and 
professional learning systems can provide state, district, 
and school leaders with essential data to ensure that all 
students have access to effective teachers and therefore 
strengthen school improvement efforts. 


States are in a unique position to build and implement educator 
evaluation and professional growth systems that districts and 
schools can then implement at scale focusing on their specific 
school improvement needs and context. Educator evaluation 
and professional growth systems can provide districts and 
schools with the data and strategies to articulate what 
effective teaching practices look like, identify areas in which 

all educators can grow and learn professionally, and provide 
educators with differentiated supports to improve their practice. 
Strengthening educator effectiveness is a key component to 
ensure equity and improvement for all students. 
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A Continued Focus on Improving Educator 
Evaluation and Professional Learning 
Systems Through the Power of Collaboration 


How can states help ensure that their systems are providing 
schools and educators with the essential data for improvement 
efforts? One powerful strategy is to collaborate with partners 
using a continuous improvement approach. Collaborating on 
continuous improvement efforts can help identify solutions, 
process strategies, share lessons learned, inspire new ideas, 
and accelerate learning and productivity. Six states have been 
doing just that—working together to investigate and address 
specific, high-priority, high-impact ways to improve their educator 
evaluation and professional learning systems. The Collaborative 
for the Continuous Improvement of Educator Effectiveness 
Systems (the Collaborative), a partnership of the Center 
on Great Teachers and Leaders (GTL Center), the West 
Comprehensive Center, and six states (Delaware, Tennessee, 


The Every Student Succeeds Act (ESSA) presents new 
opportunities for states to reaffirm, modify, or improve their 
educator evaluation and professional growth systems. ESSA 
does not require these systems but provides opportunities for 
states to continuously improve. For example, Title Il, Part A of 
ESSA authorizes states to use funds for “improving equitable 
access to effective teachers” (Section 2101(c)(4)(B)(iii) of 
Title Il of ESSA). These systems are a key component of 
equitable access by identifying which teachers need 
support and providing them differentiated professional 
learning to improve their practice. 
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Massachusetts, Colorado, Arizona, and Rhode Island), with 
support from the Northeast Comprehensive Center, have 
worked together since 2016 to address state-specific design 
and implementation challenges. Each state focused its 
Collaborative project on a problem of practice related to either 
the measures embedded in its educator evaluation system, 
the use of information from those measures, or overall system 
implementation. Across the two years, the Collaborative states 
have worked to continuously improve their systems so that 
they better assess teacher performance and provide high- 
quality, differentiated support and development. (For examples 
of the state problems of practice, see pages 10-21.) These 
states are continuing this work with the goal of improving 
teaching and learning for all students and hope to inspire 
other states and districts to do the same. 


Culminating from the work on each state’s individual, context- 
specific problem of practice, the Collaborative developed a set 
of guiding principles that the six states consider essential for 


the continuous improvement of educator effectiveness systems. 


These principles were created and informed by lessons learned 
from the participating states as they worked to solve problems 
of practice and improve their systems. Taken together, the 
guiding principles have the potential to inspire states and 
districts to leverage and cultivate their educator evaluation 
systems so that educator practice is improved. 


This brief provides an overview of the six guiding principles 
and presents short vignettes on each of the six Collaborative 
states that summarize their problems of practice, highlight 
their continuous improvement efforts within their existing 
educator effectiveness system, and draw attention to how 
the six guiding principles were infused into their work. 
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This document is meant to encourage states and districts 
that aspire to establish and continuously improve education 
evaluation systems that provide meaningful feedback and 
support to teachers. States and districts can learn from the 
guiding principles as well as from the Collaborative states’ 
lessons learned described in this report. 


As these states tackled their problems of practice and revised 
existing educator evaluation systems, the Collaborative was 
mindful to anchor the work in the evidence-based practices 
noted within empirical research and promising practices. 
Appendix A provides a brief overview of the existing and 
emerging research on educator evaluation systems and the 
features that promote the use of evaluation results to inform 
professional learning and demonstrate a positive impact on 
teacher performance and student learning. 


Interested in learning more about The Collaborative? Visit our 
webpage at https://www.sgtlcenter.org/state-collaborations 


collaborative or check out our Story Map at http://bit.ly/CollSM. 
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Why Focus on Continuous Improvement 
of Educator Evaluation and Professional 
Learning Systems? 


Over the past several years, many states and districts reformed 
educator evaluation and professional learning systems with the 
goal of improving teaching and learning for all students. The 
creation of these systems has led to many changes, including 
a greater understanding and common language concerning 
instruction (Steinberg & Donaldson, 2016) and teachers 
receiving more frequent feedback on their practice (Garet et al., 
2017; Tuma, Hamilton, & Tsai, 2018). The Collaborative 
states have experienced similar benefits and have sought to 
establish a continuous improvement cycle to strengthen their 
systems’ impact on teacher practice and student learning by 
addressing implementation challenges and by establishing 
and maintaining credibility and stakeholder buy-in. What if 
we could identify which aspects of these systems are working 
or have the potential to work with improvements, and which 
system weaknesses might need to be eliminated? One way to 
do this is through focused continuous improvement efforts. 


id ‘dl Our people are our most important asset, and we want 
to implement this comprehensive system and educator 
evaluation is a piece [of] that system. We want to make 
sure that we give the people in the system everything that 
they possibly need. So to me there isn't any choice of not 
continuously improving the system. Y Y 


Lori Renfro, Assistant Superintendent 
for Human Capital Management Systems, Maricopa County Education 
Service Agency, Arizona (a state participating in the Collaborative) 
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Continuous improvement is a cyclical approach to problem 
solving that is data-based. States and districts implementing 
this approach typically include relevant stakeholders’ reflections 
and input, identifying a problem of practice, making adjustments, 
piloting and testing potential solutions to a problem, making 
adjustments again, and then evaluating the intervention. 
Continuous improvement requires specific measurable goals, 
the flexibility to test evidence-based solutions, time to research 
and implement strategies, and collection and use of data (Best 
& Dunlap, 2014). Continuous improvement is often maximized 
when pursued in a network or community of practice. 
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Implementing educator evaluation and professional learning 
systems has been challenging work, and there is still room 
for improvement to ensure that these systems reach their full 
potential. Identifying which aspects of the evaluation and 
support system are working, or have the potential to work, and 
system weaknesses that may need to be revised or eliminated 
can inform state and district efforts to improve and support 
teacher practice. Establishing a continuous improvement focus 
not only helps strengthen systems but also sets the foundation 
for taking a system’s change approach to educator evaluation 
and planning for sustainability along the way. 
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Guiding Principles 


The Every Student Succeeds Act (ESSA) provides states 
and districts the opportunity to revisit and refocus their 
educator evaluation and professional learning systems. The 
Collaborative’s guiding principles seek to empower states 
and local education agencies (LEAs) to consider strategies to 
advance their individual educator effectiveness initiatives in 
ways that truly grow and support teachers’ practice for all 
students. The six guiding principles are as follows: 


4 


ert) Embed educator evaluation and professional learning 
systems within a coherent talent management system. 


Nz) Revisit authentic communication and collaborative 
engagement of stakeholders. 


eA 


Use data in decision making and tracking outcomes. 


Achieve consensus and clarity about system non- 
negotiables and flexibilities. 
Deprivatize the culture of teaching and feedback by 


making classroom practices, resources, and materials 
public, sharable, and storable. 


>) Model continuous improvement and learning in 
a collaborative setting. 


Improving Instruction for All Students Through Educator Evaluation as Meaningful Support | 5 


The Council of Chief State School Officers (CCSSO), the 
Measures of Effective Teaching (MET) Study, and The Aspen 
Institute have developed similar recommendations for 
improving educator evaluation systems (Bill & Melinda 
Gates Foundation, 2013; Council of Chief State School 
Officers, 2016; The Aspen Institute 2016). Collectively, 
these resources as well as the Collaborative’s guiding 
principles in this document provide states, districts, and 
other stakeholders a powerful direction and “North Star’ 
for continuously improving educator evaluation systems. 


GUIDING PRINCIPLE 1: Embed Educator 
Evaluation and Professional Learning Systems 
Within a Coherent Talent Management System 


Strong talent management systems are based on an 
understanding of what it means to be effective, the ability 
to proactively and accurately assess what this practice looks 
like in the classroom, and the capacity to support teachers 
in mastering higher levels of practice. Ideally, these systems 
are designed to cultivate needed knowledge and skills 
among entering and midcareer professionals and rely on 
implementation across the career continuum, not as isolated 


activities (Center on Great Teachers and Leaders, 2014). States 
should work across structures and organizations to ensure that 


educator evaluation and professional learning systems inform 
talent development efforts, from preservice to inservice and 
from novice teachers to seasoned teacher leaders. 
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Within state education agencies (SEAs), divisions such as 
educator effectiveness, school improvement, licensing, and 
standards and assessment should work in an interconnected 
manner. Cross-division activity intersects with the six guiding 
principles above, sharing data and using what has been 
learned to drive continuous improvement. As an essential 
equity indicator, evaluation data can be analyzed to assess 
student access to highly effective teachers and employment 
patterns of these teachers. States could consider using 
evaluation and retention data to forecast teacher needs, 
particularly in areas where shortages occur. Ultimately, data 
derived from the educator evaluation system can be linked to 
human capital reports as an important indicator of the talent 
management system’s successes or needs for improvement. 


States also should consider working with higher education 
institutions and systems to ensure that educator preparation 
and induction are aligned with and informed by the evaluation 
system. This effort typically involves data sharing across 
separate systems. 


Finally, states that are modeling use of educator evaluation 
within an overall talent management system could ensure and 
support LEAs in undertaking similar work, particularly ways in 
which educator evaluation systems could support and inform 
efforts focused on attracting, recruiting, hiring, and leveraging 
professional learning strategies and career pathways for highly 
effective teachers. 
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GUIDING PRINCIPLE 2: Revisit Authentic 
Communication and Collaborative Engagement 
of Stakeholders 


Well-designed stakeholder engagement is critical to ensure 
that the evaluation system is designed and implemented to 
support and develop teachers and leaders (Behrstock-Sherratt, 
Rizzolo, Laine, & Friedman, 2013). To achieve this end, 
states and districts need to engage educators with authentic 
communication and opportunities to contribute to system 
refinement, such as cocreating or revising evaluation 
frameworks, tools, and resources that make up the 
system. Authentic communication requires that states 
listen and respond to key stakeholders. States can gather 
input from stakeholders on what has worked well and what 
can be improved and create opportunities for stakeholders 

to co-create and help revise tools and systems. Doing this 
strengthens the evaluation system, improves implementation, 
and creates greater buy-in and support from educators. After 
engaging stakeholders and applying their feedback, states and 
districts need to be proactive and clear in communicating how 
stakeholder input was applied to any changes or adjustments 
to the system. 


States need to continually and consistently emphasize the 
purpose of the evaluation and professional learning system 
in evaluation-related communication. Moreover, they need 
to make clear linkages between evaluation and the activities 
that it is meant to support, such as professional learning 
or opportunities to advance along a career pathway. When 
states only communicate the details of implementation or 
compliance, it becomes easy for educators to view these 
systems as another accountability measure and overlook 

or lose sight of how the system has been designed to support 
teachers and improve student learning. 
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GUIDING PRINCIPLE 3: Use Data in Decision 
Making and Tracking Outcomes 


Using data can help states better understand the distribution 
of teacher effectiveness and provides a means of conducting 
continuous improvement on the overall implementation of the 
evaluation and support system in districts and schools. Most 
states have made changes to their educator evaluation 
systems in the past several years, eliminating the binary 
rating systems of the past in which 99% of teachers were 
deemed “satisfactory.” Kraft and Gilmour (2017) found 

that new evaluation systems rate more teachers in 
categories below proficient, reflecting a better distribution 
and understanding of effectiveness in the profession. But 
the study reported wide variability in teacher ratings across 
states and suggested that further work is needed to improve 
system design features and implementation practices. A 
valid distribution and understanding of teacher effectiveness 
across schools, districts, and states allows for differentiated 
supports, retention efforts, and identification of teacher 
leaders, particularly in high-need schools and districts 
where improvements are most needed. 


Given the variability in teacher effectiveness ratings and 
student growth, a close analysis of the correlation across 
the educator evaluation system metrics may inform any 
needed adjustments to both the metrics and their associated 
measures. For example, refinements to the metrics may be 
needed if teachers show high student growth scores but poor 
evaluation ratings. Likewise, an analysis of implementation 
fidelity is equally important as variability in implementation will 


impact results. In a review of state evaluation system activity, 
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the GTL Center (Menon, Berg-Jacobson, Field, & Yorke, 2015) 


found that just 13 states engaged in studies of implementation. 


These states analyzed: 
= District fidelity to the state or local model, 
= Correlation among evaluation system components, 


= Whether communication from the SEA had been 
clearly received, 


= Stakeholder satisfaction with the system, 


= Effectiveness of training and support to implement 
the evaluation system, 


= Whether unintended consequences had emerged, and 


= Impact on teacher and principal practice. 


States would do well to analyze multiple data to inform 
decisions about their evaluation systems, both to refine the 
system design and to ensure that the system is implemented 
effectively in the field. In addition, they should consider 
measuring the system against specific benchmarks that will 
promote continuous improvement. States also may help LEAs 
develop the capacity to analyze and use evaluation data to 
engage in continuous improvement efforts at the local level. 


GUIDING PRINCIPLE 4: Achieve Consensus 
and Clarity About System Non-Negotiables 
and Flexibilities 


As states work with groups of stakeholders during the 
continuous improvement process, it is vital to specify which 
aspects of the system are non-negotiable and which aspects 
have flexibility. States that identify, obtain agreement on, 
and clearly communicate aspects of the evaluation system 
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that are non-negotiable and those that are flexible establish 
transparency and gain a clearer understanding of expectations, 
both internally at the SEA and for the LEAs and schools 
implementing the system. Leadership would do well to ensure 
that the non-negotiables are clearly understood and to clarify 
where flexibility is permitted, so that decisions are based on 


the available evidence base and informed by key stakeholders. 


States take a variety of approaches to the state-local 
relationship in evaluation systems, such as a single state 
system, a menu of approved systems, a state framework 
with components for districts to determine, and the ability 
of districts to establish their own systems (Gandha & Baxter, 
2016). Regardless of approach, the state should be clear on 
a research- and standards-based structure that gives districts 
enough flexibility to customize the implementation of the 
system to their context while giving the state meaningful and 
comparable data to analyze. To determine which approach is 
appropriate, states need to consider the system’s goals and 
desired outcomes along with feedback from key stakeholders. 


Guiding Principle 5: Deprivatize the Culture of 
Teaching and Feedback by Making Classroom 
Practices, Resources, and Materials Public, 
Sharable, and Storable 


For evaluation systems to successfully inform individual 
and collective improvements in professional practice, local 
educators must be engaged in sharing, accessing, and 

analyzing examples of their own work and the work of their 
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peers. States need to promote and foster the collaborative 
sharing of effective instructional practices, resources, and 
materials. States can make practices, resources, and materials 
shareable and available publicly by providing vetted, curated 
sets of resources that districts can use, apply, or adapt to 
meet their local needs. The design and implementation of 
the evaluation system itself can lead to iterative and action- 
oriented feedback. To create a culture and system that uses 
and relies on shared feedback, examples of practice, and 
resources and materials, states must first analyze current 
policies, systems, strategies, and tools to assess whether 
they would support such a system. States could solicit 
stakeholder input to understand revisions that could be 
made to the system to make such sharing feasible and 
usable. States should create and/or revise policies, systems, 
strategies, and tools that increase effectiveness and growth 
for educators through the collaborative sharing and storing 
of effective instructional practices, resources, and materials. 
States should engage and support LEAs to provide teachers 
with time to work collaboratively, to create hybrid teacher 
leader roles, and to observe one another’s practice. 


AS NOTED BY RICHARD ELMORE, 


4 4 Privacy of practice produces isolation; isolation is the 
enemy of improvement. y 9 


Elmore (2000). Building a New Structure for School Leadership. 
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Guiding Principle 6: Model Continuous 
Improvement and Learning in a 
Collaborative Setting 


States should track the efficacy of their educator effectiveness 
systems over time and use that data to inform the continuous 
improvement of these systems. Data should be collected 
at multiple points in the year to reflect upon short-term 
outcomes, and states should use those results to drive 
system improvement. Similarly, states might consider using 
the data generated by these systems to determine which 
students are and are not afforded equitable access to 
effective teachers as part of efforts to ensure equitable 
access to excellent teachers. 


States should model this process for districts and support the 
use of continuous learning of evaluation and support systems 
at the LEA level. Furthermore, states should consider it their 
role to promote continuous improvement and learning in 
evaluation and to support systems by collaborating across 
the state and LEAs. 


Improving Instruction for All Students Through Educator Evaluation as Meaningful Support | 9 


The following section highlights the work of the Collaborative 
states to improve their evaluation and educator effectiveness 
systems by depicting how the guiding principles were applied. 
The six vignettes that follow reflect specific problems of 
practice with identified strategies related to one or more 
of the Guiding Principles, with a focus on improving the 
evaluation and educator effectiveness system. 


Review our self-assessment to help determine which guiding 
principle may be the best entry point for continuously 
improving your educator evaluation and professional growth 
system. Find the self-assessment at https://www.gtlcenter. 


org/sites/default/files/Collaborative Selfassessment.pdf 
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ARIZONA: Strengthening Evaluator Feedback 


Passed in 2011, Arizona’s educator 
evaluation law requires districts 

and charter school entities to align 
their evaluation systems with the 
Arizona State Board of Education’s 
adopted framework. This framework 
provides LEAs with the flexibility needed 
to design their own evaluation models, adopt existing models, 
or implement the optional model designed by the Arizona 
Department of Education’s Effective Teachers and Leaders Unit. 


Over the past 3 years, ADE collected and analyzed aggregate 
teacher evaluation scores and compared them with student 
performance data, uncovering a disconnect between the two. 
The data showed that 94% of teachers were rated Effective or 
Highly Effective, yet only 39% of students passed the English 
language arts and math portions of the state assessment. 
Many states have experienced a similar challenge of little 
differentiation in teacher evaluation results; most teachers 
were rated Effective or higher (Doherty & Jacobs, 2015). 
Using these data in decision making and tracking outcomes, 
ADE adopted two strategies to address this problem of practice: 


= Increased the capacity of principals to conduct 
meaningful observation and feedback cycles through 
the development of ADE’s Qualified Evaluator Academy. 
To strengthen teacher feedback from principals or 
supervisors of teachers, ADE developed the Qualified 
Evaluator Academy (QEA) in 2016-17. The goal was to 
strengthen the practice of teacher evaluators to obtain 
a more accurate review of teacher performance to 
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better align teacher evaluation Summative scores 
with student academic progress. In Year 1 of the 
Collaborative, six cohorts, with approximately 

200 participants, successfully completed the 5-day 
academy. Using a pre- and post-assessment with a 
Likert scale from 1 to 5, participants increased their 
knowledge of evaluations, from an average of 3.0 to 
4.25 after completion of the academy. In addition, 
98% of the participants responded that the QEA either 
“Met” or “Exceeded” their expectations. To continue 
addressing the problem of practice, three additional 
cohorts were added for Year 2 of the Collaborative. 
ADE is in the process of assessing the impact of the 
QEA on educator evaluation through a case study with 
past LEA participants. 


Aligned ADE efforts across the agency on joint 
projects pertaining to educator evaluations. 

A concerted effort across divisions and programs 
within ADE has been made to “break down the silos.” 
The QEA included cross-agency planning and 
implementation, specifically addressing areas in 
special education, English language learners, arts, 
physical education, Title |, career and technical 
education, and early childhood education. Participants 
conversed with specialists from each area, focusing 
on student engagement, instructional strategies, 
and the collection of evidence. Talent management 
has become a focus area in the state’s school 
improvement and turnaround efforts. 


Introduction 
Guiding Principles 

5 Arizona 
Colorado 
Delaware 
Massachusetts 
Rhode Island 
Tennessee 
Appendix A 


References 


For the second phase of the Collaborative, ADE plans to 
focus on supporting evaluators of principals. Using a survey, 
the agency has collected data on the level of interest and 
need for a similar training for evaluators of principals. 


Throughout the Collaborative, ADE focused on continuously 
improving its evaluation and professional growth system by 
improving principals’ capacity to conduct observations and 
provide high-quality feedback to teachers through the QEA. 
Other states seeking to strengthen their systems may consider 
similar strategies of using data in decision making and modeling 
continuous improvement in a collaborative setting. 
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PRINCIPAL QUOTES 


4 4 I cannot imagine evaluating teachers without this 


training. 9 Y 


4 4 This is the most I've learned about evaluations... 9 9 


é & I feel so much more prepared and knowledgeable about 
the whole process. 9 9 


To see the latest updates about Arizona’s project, visit 
https://www.gticenter.org/state-collaborations/collaborative 


arizona-profile. 
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COLORADO: Continuous Improvement on the Teacher Rubric 


In 2010, Senate Bill 10-191 
passed, changing the way 

all educators are evaluated 

in Colorado with the goal of 
improving educators’ practice 
and student learning. The 
Colorado Department of Education (CDE) conducted a 5-year 
pilot of the evaluation system and collected and examined 


In addition, CDE conducted 36 focus groups across 
the state to share the intended changes and learn 
whether practitioners thought these revisions would 
result in more meaningful feedback to educators. The 
input gained from focus groups went back to the TWG 
for further revisions. The revised rubric was piloted 
during the 2017-18 school year in 50 districts and 
Boards of Cooperative Educational Services (BOCES) 


Introduction 


system data for decision making and tracking outcomes. with feedback loops on the revisions to determine Guiding Principles 
CDE analyzed qualitative and quantitative data in a variety of whether the updates allowed more time for meaningful 

ways: a general inbox to collect information broadly, baseline feedback due to clarification of language and reduction ines 

and end-of--year surveys, and an analysis of ratings that of redundancies. 5 Cdlorado 
identified high correlations between elements, confirming = Learned from districts piloting the newly revised rubric. 

the qualitative evidence of redundancies in the instrument. CDE has five regional specialists based throughout the Delaware 
The data collection and analysis highlighted that the lack state who are charged with introducing the new rubric Y : 

of quality feedback to educators was a significant barrier to focally and prévidine Suppertte te-pllotaisinietsand ee 
implementation by showing that the length and redundancy BOCES, and eventually to all districts and BOCES in ahedeisiand 
of the evaluation instrument was problematic. their region. These regional specialists do not provide 

CDE’s problem of practice for the Collaborative was to ensure training but instead coach districts to think about how eee 
that evaluators have time to provide teachers with quality best to support educators with the shift in the revised Appendix A 


feedback and instructional coaching supports. CDE adopted 
the following strategies: 


= Revised the teacher rubric through the analysis of 
data with stakeholders. During the 2016-17 school 
year, CDE formed a technical working group (TWG) 


comprised of district, school, and teacher representatives 


to lead the rubric revisions. This group examined data 


4 4 These changes were about the field, they werent just about 


CDE just deciding to make the change. They were not 
only pushed by the field, but generated by the field and 


then confirmed by the field. yy 


References 


Courtney Cabrera, Educator Effectiveness Manager, 


to make informed decisions about needed revisions to ° 
Colorado Department of Education 


reduce redundancies and add clarity within the rubric. 
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rubric. In addition, during a meeting of pilot districts, 
CDE organized district teams by region to discuss and 
share challenges and solutions related to implementing 
the revised rubric. The regional organization ensured 
that smaller rural districts could brainstorm solutions 
together to address similar implementation challenges. 


For the second phase of the Collaborative, CDE plans to revise 
the State Model Evaluation System for Principals/Assistant 
Principals. The goals of this revision are to ensure greater 
quality of feedback provided to principals, boost instructional 
practices for teachers, and improve alignment with the revisions 
to the teacher evaluation system. CDE plans to adopt similar 
strategies for revisions to the principal evaluation instrument 


using authentic stakeholder engagement through a TWG 
and focus groups. 


Improving Instruction for All Students Through Educator Evaluation as Meaningful Support | 13 


Throughout the Collaborative, CDE focused on strategies 
to improve the quality of feedback teachers received by 
revising the evaluation rubric and providing support and 
coaching on the new rubric. States and districts interested 
in continuously improving their evaluation systems may consider 
a similar strategy of using data for decision making and 
engaging stakeholders to help identify areas to continuously 
improve their evaluation and professional growth systems. 


To see the latest updates about Colorado’s project, visit 


https://www.gtlcenter.or; 


colorado-profile. 


state-collaborations/collaborative 
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DELAWARE: Communication and Stakeholder Engagement 2.0 


Based on perception data from a statewide 
educator survey, educator interviews, and 
regular monitoring, the Delaware Department 
of Education (DDOE) learned that there was 
misinformation and confusion surrounding the 
evaluation system, its purpose, and how it can 
support teacher professional growth. The 
survey also illustrated that there was a lack of buy-in from 
educators on the evaluation system. DDOE staff identified 
that less-than-effective communication from the department 


perhaps led to weak communication from administrators to 
educators. This miscommunication led to poor understanding 
and negative perceptions of the evaluation system by educators. 
In addition, department staff recognized during monitoring 
visits that communicating the correct information and clarifying 
the evaluation system's ability to support professional learning 
increased teacher buy-in. This direct feedback from stakeholders 
in the field helped identify that Delaware needed to revisit 
authentic communication. When Delaware rolled out its 
teacher evaluation system, the communication strategy 
focused more on the details of the new appraisal process 
and did not emphasize how the system could improve educator 
practice and student learning. 


As part of the Collaborative, DDOE’s problem of practice was 
how to strengthen and improve communication strategies so 
that educators would receive accurate information and increase 
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educator buy-in. Delaware adopted three strategies to improve 
communication on the evaluation system: 


= Offered an administrator refresher training focused 


more on improving practice and less on ensuring 
compliance. DDOE provided a 1-day training to all 
administrators in the state during the summer of 2017. 
The training not only revisited policy updates but also 
highlighted the intent of the system and showed how 
improving educator practice is evident throughout all 
the practices in the system. Administrators learned 
how to use evaluation data to drive professional 
learning and quality goal-setting strategies. 


Established quarterly Delaware Performance Appraisal 
System (DPAS II) lead meetings that include an 
assigned lead liaison for the evaluation system from 
each district. DDOE communicates regularly with this 
group of liaisons through e-mail on major changes, 
events, and training. In addition, DDOE meets with 
this group in-person and virtually to share and clarify 
information and teach best practices for monitoring 
the quality of evaluations, collecting and analyzing 
data, calibrating, and providing training. The group has 
opportunities during these meetings to share helpful 
tips, Successes, and strategies to address problems 
of practice. For new teachers and as a refresher 
for experienced teachers, DDOE created a “Student 
Improvement Component” and a “DPAS II: Laying the 
Foundation” professional development session that 
the liaisons could deliver. 
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= Launched a monthly newsletter on the evaluation the system so that educators receive frequent, timely, and 


system framework and processes. Each edition of the targeted feedback on their practice. To address these issues, 
newsletter* contains research behind selected criteria DDOE plans to engage a group of diverse stakeholders from 
from the teacher evaluation rubric and notices and across the state to work on continuously improving the 

links to professional development sessions that evaluation system. 


administrators can use to dig deeper into the criteria 
) eP6) Over the course of its work during the Collaborative, DDOE 


by examining the elements of the criteria, identifying 
i ‘i . : engaged in authentic communication with stakeholders, 
common “look-fors,” and observing effective = SS eee 
re addressed misconceptions of the educator evaluation system 
performance of the criteria. ea ahaa te a a att ete 
to improve buy-in and implementation, and plans to further 
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For the next phase of the Collaborative, DDOE plans to authentically engage stakeholders to continuously improve 

examine whether the current evaluation tool clearly reflects the teacher evaluation rubric. Other states and districts may Guiding Eipelples 

rigorous teaching practices aligned to the InTASC standards consider modeling similar strategies to improve communication peer 

and shifts in the Common Core State Standards. DDOE plans and engage stakeholders to continuously improve their 

to address whether there are opportunities to streamline educator evaluation and professional growth systems. Colorado 

the rubric, eliminate redundancy, and focus on the critical 

indicators of performance that can improve teaching. In To see the latest updates about Delaware's project, visit 5 Delaware 

addition, department staff plan to explore how to improve https://www.gticenter.org/state-collaborations/collaborative Massachusatts 
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+ For examples of monthly newsletters, see https://www.doe.k12.de.us/ 


Page/3742. 
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MASSACHUSETTS: A Commitment to Student Learning as One of Multiple 


Measures in the Evaluation System 


The Massachusetts Educator 
Evaluation Framework originally 
included a Summative 
Performance Rating of a 


a 3 


teacher’s effectiveness across 
four standards of practice, as well as a separate Student 
Impact Rating based on multiple measures of student 
learning, growth, and achievement. Through conversations 
with a variety of stakeholders, the Massachusetts Department 
of Elementary and Secondary Education (DESE) found that 
the separate Student Impact Rating was a concern for many 
districts. To identify the specific challenges, DESE collected 
self-reported data from districts on the barriers to establishing 
Student Impact Ratings for educators. The data showed that 
some districts were struggling to implement the Student 
Impact Rating due to a lack of data and assessment literacy, 
particularly in non-tested grades and subjects, making 

it difficult to ensure rigor and parity for all educators. In 
addition, the state faced pushback from educators on 
having a significant portion of a teacher’s evaluation 
comprised of this separate rating. 


Massachusetts’s problem of practice as part of the 
Collaborative was how to revise the separate impact rating 
in a way that addressed stakeholders’ concerns yet still 
yielded information about a teacher’s impact on student 
learning, growth, and achievement. For Massachusetts, 
the balance of non-negotiables and flexibility included 
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maintaining evidence of student learning as a non-negotiable, 
while allowing for flexibility in the process used to incorporate 
these measures into an educator’s evaluation. DESE adopted 
two strategies to address this problem of practice: 


1. Identified core principles of including student learning 
in educator evaluation via authentic stakeholder 
engagement. The state convened a group of 
superintendents and union representatives to provide 
feedback and ideas on how to measure student learning 
and how to integrate that information into an educator’s 
evaluation. Some of the superintendents’ feedback 
was that the separate Student Impact Rating was 
redundant and that teachers were already examining 
student data in the other standards within the 
Massachusetts educator evaluation framework. 


2. Revised regulation language on the student impact 
rating. With input from stakeholders and support from 
the Collaborative, DESE drafted revised regulation 
language that eliminated the Student Impact Rating 
and embedded evidence of impact on student learning 
into the second standard of the performance evaluation. 
Having a separate impact rating meant that educators 
and districts were looking at teacher practice and 
the impact on student learning separately. This 
revised model streamlined the process while reinforcing 


conversations on how practice can impact student learning. 
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During the next phase of the Collaborative, DESE will focus 

on how to support districts in making good human capital 
decisions. The Educator Evaluation Framework can support 
every stage of the human capital process, including preparation, 
recruitment, induction, professional learning, and recognition. 
DESE plans to examine what the state is doing in each 
stage, what is and not working within LEAs, what could be 
done to improve coherence within each stage, and how the 
Educator Evaluation Framework can be a driver of effective 
talent management. 


Throughout its work with the Collaborative, DESE identified 
its non-negotiable policies and engaged stakeholders in 
continuously improving the measurement of student learning 
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within its teacher evaluation system. Identifying what policy 
areas are non-negotiable or flexible for a state education 
agency is a great early step for other states and districts to 
consider prior to soliciting stakeholder feedback on how to 
improve the evaluation system. 


To see the latest updates about Massachusetts’ project, visit 
https://www.gticenter.org/state-collaborations/collaborative 


massachusetts-profile. 
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RHODE ISLAND: Reimagining Student Learning as Part of the Educator 


Evaluation System 


The Rhode Island Department of 
Education (RIDE) identified student 
learning objectives (SLOs)—the state’s 
measure of student learning—as an 
area to work on improving within the 
educator evaluation system. RIDE 
collected feedback on the teacher 
evaluation system from educators across the state for 6 years. 
The feedback from educators showed that the SLO process 
may not be as well connected to what teachers are doing 
instructionally and, therefore, may not have the intended 


impact of improving instructional practice. In addition to the 
survey, RIDE held a summit with a variety of stakeholders 
focused on the evaluation system. At that summit, multiple 
stakeholders came together to propose a set of weights and 
scoring bands that could be used across all state-approved 
teacher evaluation models. Unfortunately, participants were 
not yet prepared to move toward a common set of weighted 
points and cut-score ranges. Instead, they indicated the 
more pressing need to address the concerns with SLOs. 


Based on this feedback, RIDE’s problem of practice for 
the Collaborative was to evolve the measurement of student 
learning to reflect a more authentic and accurate experience 
for educators across Rhode Island. RIDE adopted the 
following strategies: 


= Engaged in authentic communication and 


collaborative stakeholder engagement. RIDE staff 
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wanted to ensure that the new models of measuring 
student learning would help improve instructional 
practice and gain buy-in from educators. To do this, 
RIDE launched a collaboration with one of the state’s 
teacher unions and two teacher representatives from 
the two most widely used evaluation models. State 
leadership and teacher union representatives have 
participated in every meeting and all stakeholders 
have provided feedback throughout the reimagining 
process. Employing authentic communication to the 
field was critical, so that all educators, not just those 
participating in the meetings, were informed on the 
upcoming changes. Some of the communication 
strategies included entries in the Commissioner’s 
weekly field memo to inform educators of RIDE’s 
commitment to reimagining student learning through 
evaluation. Each message to the field, whether through 
the Commissioner’s field memo, information sessions, 
or support sessions, was vetted with the collaborative 
stakeholder group and modified accordingly. By 
using multiple communication and engagement 
strategies, RIDE intends to increase transparency 
regarding the improvements and shifts made to the 
evaluation system. 


Conducted a SWOT analysis. RIDE did not want to 
completely disregard all the work and lessons learned 
from implementing SLOs. The Rhode Island team 
conducted a SWOT analysis in the fall of 2016 to 
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identify strengths, weaknesses, opportunities, and 
threats (SWOT) to determine what about SLOs was 
working well and what was not working. This analysis 
helped Rhode Island build upon the previous work and 
focus on continuous improvement. 


= Developed guiding principles. Based on the SWOT 
analysis and data collection, the Rhode Island team 
developed a list of guiding principles that would help 
frame the new models of measuring student learning 
and established the non-negotiables and flexibilities. 
The guiding principles included: 


iii. Students: A representative class or group of 
students is included. 


iv. Standards: Content standards are appropriate 
and prioritized to the course. 


v. Time frame: Both long-term and short-term cycles 
are encouraged. 


vi. Evidence: Multiple, varied sources of standards- 
aligned evidence are used. 


vii. Strategies: Instructional strategies are planned 
for and supported. 


viii. Expectations: Student learning expectations are 
articulated and monitored. 


From this work, RIDE selected three models of measuring 
student learning to pilot and collect feedback in the 2017-18 
school year to determine the viability of formally offering these 
models statewide. The three models were initially described 
as follows: 


= Embedded Practice Model: This model prioritizes 
teachers using measures of student learning that are 


already in use in a teacher's day-to-day classroom. The 
teacher has ongoing data discussions with colleagues 
and evaluators to drive instructional decisions related 
to the prioritized content area. 


= Portfolio Model: The teacher submits work samples 
from three students representing varied skill levels 
to provide evidence of student learning over time. 


= Student Learning Goals Model: Similar to SLOs, 
teachers create a long-term or short-term student 
learning goal but have the flexibility to adjust instruction 
based on student data collected throughout cycles 
of instruction. 


RIDE plans to continue to engage stakeholders, particularly 
educators participating in these pilots, to gather feedback 
on how to strengthen and improve the three models and 
ultimately determine each model’s viability as a future 
statewide model for measures of student learning. 


Over the course of the Collaborative, RIDE engaged in 
authentic communication and stakeholder engagement to 
identify its problem of practice and to develop new models 

of measuring student learning. Other states may consider a 
similar approach of engaging stakeholders to identify areas of 
improvement within their educator evaluation and professional 
growth systems and to develop new strategies or approaches 
within their systems. 


To see the latest updates about Rhode Island’s project, visit 
https://www.gticenter.org/state-collaborations/collaborative 


rhode-island-profile. 
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TENNESSEE: Deprivatizing the Culture of Teaching and Feedback Through 


Student Growth Portfolios 


In 2011, Tennessee 
implemented a statewide 


the Tennessee Educator Acceleration Model (TEAM), which 
consists of frequent observations, feedback, and student 
growth measures. The Tennessee Department of Education 
(TDOE) collects feedback on TEAM and its implementation 
using an annual educator survey. According to survey results, 
teachers in Tennessee wanted more frequent and specific 
feedback on their performance. To address this response, 
TDOE developed the student growth portfolio model—a 
student growth measure for teachers in nontested grades 
and subjects. This measure helps deprivatize the culture of 
teaching and feedback by making content-based practices, 
materials, and resources public, shareable, and storable. 
Teachers upload their student work samples onto an online 
platform where they are reviewed, scored, and given feedback 
by a peer evaluator. In addition, there are tools, resources, 
guidance documents, and videos accessible to teachers 
through this platform. The models are currently available for 
teachers of physical education, fine arts, world languages, 
prekindergarten, kindergarten, and first grade. 


During the 2016 legislative session, the Tennessee General 
Assembly passed the Pre-K Quality Act, which requires all 
districts to utilize a student growth portfolio model for their 
prekindergarten and kindergarten teachers. Tennessee was 
faced with 5,500 teachers and their districts needing training 
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teacher evaluation system, 


to implement this new requirement. How to address expanding 
the student growth portfolio model became Tennessee's 
problem of practice for the Collaborative. TDOE implemented 
three key strategies to address this problem: 


= Ensured alignment between what the portfolios 
capture and the revised English language arts and 
mathematics standards. 


= Developed and facilitated district trainings on the new 

standards and the portfolio model. For Tennessee, it 
was not possible to train all 5,500 prekindergarten 
and kindergarten teachers on the new standards and 
the portfolio model. Providing district trainings using 

a train-the-trainer model allowed the state to have 

a consistent message and give districts increased 
ownership in the process. 


= Communicated changes within the student growth 
portfolio model, such as implementing the new 
online platform. 


To implement the student growth portfolio model successfully, 


TDOE focused on balancing policies that were non-negotiable 


and flexible for districts. For example, one non-negotiable is 
that every district is required to provide a specific number 
of peer evaluators who are trained to review and score the 
portfolios. However, districts have the flexibility to determine 
who will serve in this role. Often, the non-negotiables are 
determined by statute or stakeholder input. For the student 
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growth portfolios, TDOE established a statewide portfolio 
council comprised of teachers currently using portfolios, 


district portfolio liaisons, and prekindergarten and kindergarten 
teachers who helped inform which items were non-negotiable 


or flexible. For Tennessee, stakeholder input from a statewide 
council was helpful to drive the development of the separation 
between non-negotiables and flexible items and to gather 
feedback for any midcourse corrections. In addition, it is 
important that the non-negotiables are critical items that 
connect to achieving the state’s outcomes to avoid 
micromanaging districts. 


For the second phase of the Collaborative, TDOE’s Office 

of Educator Effectiveness plans to examine strategies to 
prompt leadership at the department to integrate initiatives 
such as response-to-intervention, professional learning 
systems, human capital management, and accountability 
for the TEAM model with a group of pilot districts during 
the 2019-20 school year to build greater coherence. 
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Over the course of the Collaborative, TDOE expanded a student 


growth measure that strengthened feedback provided to 
teachers and supported deprivatizing the culture of teaching 
and learning. Other states considering how to continuously 
improve their teacher evaluation and professional growth 
systems may want to identify strategies for storing and 
sharing resources related to high-quality teaching and how 
to provide teachers with specific feedback. 


To see the latest updates about Tennessee’s project, visit 
https://www.gtlcenter.org/state-collaborations/collaborative/ 


tennessee-profile. 
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Appendix A 


Evidence of Impact Research 


Although district and state teacher evaluation reform has been 
under way for nearly a decade, the number of definitive studies 
on the impact of these systems on teacher practice or student 
learning is limited. Dee and Wyckoff’s (2013) study examined 
the District of Columbia’s IMPACT evaluation system and found 
that linking multiple measures of teacher performance with 
substantial incentives can improve the performance of the 
teaching workforce. A 2012 study of Cincinnati’s Teacher 
Evaluation System found that teacher performance and 
student outcomes improved as a result of the system, which 
used a practice-based assessment relying on multiple, highly 
structured classroom observations conducted by experienced 
peer teachers and administrators (Taylor & Tyler, 2012). 


One of the most significant studies of teacher evaluation was 
the 3-year Measures of Effective Teaching (MET) study by the 
Gates Foundation (2010-13; Cantrell & Kane, 2013). Findings 
from the Gates Foundation MET study include the following: 


= Observations and student surveys can provide teachers 
with useful feedback on practice. 


= Strategies to increase teacher trust in evaluation data 
include rigorous observer training and certification, the 
use of multiple observers, and student confidentiality 
when administering student surveys. 


= Multiple measures are important; value-added, 
observation, and student survey data can provide 
important information about teacher practice. 
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= Multiple measures should be combined in a balanced 
way to be most reliable and provide the most useful 
information. 


= Videos can provide teachers with information on 
practice and can provide observers with training and 
certification opportunities. 


Evidence From Implementation Research 


Several states and districts have evaluated the implementation 
of their teacher evaluation systems. Reports from these 
studies provide information on the elements of teacher 
evaluation systems that have been successful as well as 
lessons learned. Some common themes emerged across 
these reports, highlighting possible attributes of successful 
teacher evaluation systems across three areas: implementing 
strong measures to assess teacher performance, enhancing 
the observation and feedback process, and using evaluation 
results to inform professional development and support. 


Implementing Strong Measures to Assess 
Teacher Performance 


Everyone involved in the evaluation process should feel 
confident in the quality of the data and the capacity of the 
system to successfully capture the complexity of teaching 
(Milanowski et al., 2014). It is generally understood that to 
accurately assess teachers’ performance for the purposes of 
providing feedback to improve, multiple measures are needed 
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(Cantrell & Kane, 2013). The two most commonly used 

and examined measures include student growth data and 
observations of teacher practice, although researchers and 
practitioners also have examined measures Such as lesson 
plan reviews, teacher self-assessments, measures of 
professional learning, student artifacts, teacher portfolios, 
and student surveys, which all can provide a reliable source 
of data on classroom climate, more reliable than classroom 
observation in some cases (Cantrell & Kane, 2013; Hull, 
2013). Student growth data, through value-added measures 
and/or SLOs, provide important information on student 
learning (Hull, 2013; Lacireno-Paquet, Morgan, & Mello, 
2014). Goe, Wylie, Bosso, and Olson (2017), in their recent 
report based on State Teachers of the Year survey data, 
say that states should consider ways to measure teachers’ 
contributions to student learning rather than focusing on 
the results from a single standardized assessment. That 
assessment data can help educators identify in what areas 
students need additional support instead of serving as a 
weighted percentage of an evaluation score. Similarly, some 
teacher respondents from the 2016 teacher evaluation 
implementation study and principal respondents from a 
2017 study criticized the use of student achievement data to 
evaluate their performance—they expressed concern that the 
selected tests (especially the state assessments) do not fully 
capture their students’ actual growth or are not aligned with 
the school curriculum (Anderson, Butler, Palmiter,& Carcaira, 
2016; Derrington, 2017). 


Observations of teacher practice provide the opportunity for 
teachers to receive ongoing and actionable feedback on their 
practice. According to a recent study by Goe, Wylie, Bosso, 
and Olson (2017), educators trust in and want to continue 
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classroom observations for both teacher evaluation and 
professional development, and they value both formal and 
informal observations as well as walk-throughs. Multiple 
observations should be part of a teacher's evaluation cycle, 
performed by more than one observer for increased reliability 
(Cantrell & Kane, 2013; Curtis, 2012; Donaldson et al., 2014; 
Kane & Staiger, 2012). Ideally, at least one observation 
should be conducted by a trained observer from outside the 
teacher’s school (Whitehurst, Chingos, & Lindquist, 2014). 


Measuring student growth has been the most challenging 
aspect in implementing teacher evaluation systems. For many 
states and districts, statewide assessments are applicable 
only to teachers in certain grades and subjects. Valid and 
reliable alternative assessment options for teachers of 
nontested grades or subjects are limited, and alternative 
growth measures, such as SLOs, are time-consuming 
and require stronger data and assessment literacy skills 
for teachers (Lachlan-Hache, Cushing, & Bivona, 2012; 
McCullough, English, Angus, & Gill, 2015). Seven districts 
in the 2016 teacher evaluation implementation study used 
SLOs for measuring teacher impact on student performance 
and highlighted challenges, such as setting realistic and 
consistent goals for measuring student growth and ensuring 
that principals give teachers fair and consistent advice 
on what the SLOs should be and how to measure them 
(Anderson et al., 2016). 


Districts should continuously collect and analyze data on 
teacher evaluation scores and the scores of teachers on 
each measure to monitor reliability (Kane & Staiger, 2012) and 
to analyze connections between teacher scores and student 
characteristics (Jiang & Sporte, 2016; Whitehurst et al., 2014). 


Introduction 
Guiding Principles 
Arizona 
Colorado 
Delaware 
Massachusetts 
Rhode Island 
Tennessee 

5 Appendix A 


References 


Enhancing the Observation and Feedback Process 


There are certain teaching practices, such as structuring, 
planning, asking lower and higher order questions, and providing 
feedback, which have been shown to account for significant 
improvements in student learning (Araujo et al., 2016; Bold 
et al., 2017; Cantrell & Kane, 2013). To continuously support 
teachers’ ability to practice these and other high-quality 
instructional strategies, observation and feedback systems 
must improve, including the depth and quality of training 

and support that evaluators receive. Studies suggest that 
principals and teachers need significantly more training and 
support than they have had in the past to learn how to have 
meaningful conversations about improving instructional 
practice (Curtis, 2012; Donaldson et al., 2014). Teachers 
from Goe et al.’s study (2017) stressed the importance of 
focusing more on targeted feedback for professional growth 
and improving instruction and deemphasizing the evaluation 
“score.” They also suggested the importance of providing time 
and resources for informal peer observation and discussion, 
not only the formal process (Goe et. al., 2017). Gandha and 
Baxter (2016) suggested three strategies to support better 
feedback to teachers: (1) encouraging teacher-led dialogue, 
such as asking teachers to share examples of their impact 
on student learning; (2) creating opportunities for teachers 

to practice feedback with each other during, for example, 
peer observations or professional learning communities; 

and (3) linking feedback to collective goals. 
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Using Evaluation Results to Inform Professional 
Development and Support 


Improving professional learning systems for teachers is 
essential. TNTP’s 2015 Mirage report states that, “School 
systems are not helping teachers understand how to improve— 
or even that they have room to improve at all” (TNTP 2015; 
p. 2). Systems designed to assess teacher performance— 
precisely where teachers’ instructional practice is strong and 
where it could use improvements—should be linked to 
professional learning clearly and consistently. Individual 
evaluation conversations should include opportunities for 
useful feedback and coaching (Curtis, 2012; Shakman, Zweig, 
Bocala, Lacireno-Paquet, & Bailey, 2016). The overall system 
should have clear links for individual and group professional 
development opportunities based on evaluation system 
results (Milanowski et al., 2014). Still, research from 
TNTP (2015) concluded that it may be difficult to identify 
professional learning opportunities that improve teacher 
practice. Yet, states and districts may be able to learn from 
others who have already tied evaluation results to professional 
development and support (See Shakman et al., 2016, for 
example). Furthermore, the Goe et. al. (2017) study 
recommends that schools, districts, and states lean 

on professional development that is differentiated and 
individualized for teachers, such as online self-paced study 
opportunities, watching and discussing videos of excellent 
teaching with colleagues, and collaborating with teachers 
who have similar interests and needs for professional growth 
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or who are from the same content or grade level. States have create an online portal that provides professional learning 


the opportunity to provide resources and support to districts resources aligned to specific teaching practices in state 
to help them connect how to use evaluation data to inform standards or observation rubrics, or develop online or 
professional learning opportunities. For example, states can blended learning professional learning modules (Bill 
provide in-person trainings for principals or evaluators on & Melinda Gates Foundation, 2013). 


evaluation feedback and coaching for professional growth, 
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